Unsupervised Light Field Depth Estimation via Multi-view Feature Matching with Occlusion Prediction

Depth estimation from light field (LF) images is a fundamental step for some applications. Recently, learning-based methods have achieved higher accuracy and efficiency than the traditional methods. However, it is costly to obtain sufficient depth labels for supervised training. In this paper, we propose an unsupervised framework to estimate depth from LF images. First, we design a disparity estimation network (DispNet) with a coarse-to-fine structure to predict disparity maps from different view combinations by performing multi-view feature matching to learn the correspondences more effectively. As occlusions may cause the violation of photo-consistency, we design an occlusion prediction network (OccNet) to predict the occlusion maps, which are used as the element-wise weights of photometric loss to solve the occlusion issue and assist the disparity learning. With the disparity maps estimated by multiple input combinations, we propose a disparity fusion strategy based on the estimated errors with effective occlusion handling to obtain the final disparity map. Experimental results demonstrate that our method achieves superior performance on both the dense and sparse LF images, and also has better generalization ability to the real-world LF images

Event Encryption: Rethinking Privacy Exposure for Neuromorphic Imaging

Bio-inspired neuromorphic cameras sense illumination changes on a per-pixel basis and generate spatiotemporal streaming events within microseconds in response, offering visual information with high temporal resolution over a high dynamic range. Such devices often serve in surveillance systems due to their applicability and robustness in environments with high dynamics and strong or weak lighting, where they can still supply clearer recordings than traditional imaging. In other words, when it comes to privacy-relevant cases, neuromorphic cameras also expose more sensitive data and thus pose serious security threats. Therefore, asynchronous event streams also necessitate careful encryption before transmission and usage. This letter discusses several potential attack scenarios and approaches event encryption from the perspective of neuromorphic noise removal, in which we inversely introduce well-crafted noise into raw events until they are obfuscated. Evaluations show that the encrypted events can effectively protect information from the attacks of low-level visual reconstruction and high-level neuromorphic reasoning, and thus feature dependable privacy-preserving competence. Our solution gives impetus to the security of event data and paves the way to a highly encrypted technique for privacy-protective neuromorphic imaging

LRT: An Efficient Low-Light Restoration Transformer for Dark Light Field Images

Light field (LF) images containing information for multiple views have numerous applications, which can be severely affected by low-light imaging. Recent learning-based methods for low-light enhancement have some disadvantages, such as a lack of noise suppression, complex training process and poor performance in extremely low-light conditions. To tackle these deficiencies while fully utilizing the multi-view information, we propose an efficient Low-light Restoration Transformer (LRT) for LF images, with multiple heads to perform intermediate tasks within a single network, including denoising, luminance adjustment, refinement and detail enhancement, achieving progressive restoration from small scale to full scale. Moreover, we design an angular transformer block with an efficient view-token scheme to model the global angular dependencies, and a multi-scale spatial transformer block to encode the multi-scale local and global information within each view. To address the issue of insufficient training data, we formulate a synthesis pipeline by simulating the major noise sources with the estimated noise parameters of LF camera. Experimental results demonstrate that our method achieves the state-of-the-art performance on low-light LF restoration with high efficiency

Cross-Camera Human Motion Transfer by Time Series Analysis

Along with advances in optical sensors is the increasingly common practice of building an imaging system with heterogeneous cameras. While high-resolution (HR) video acquisition and analysis benefit from hybrid sensors, the intrinsic characteristics of multiple cameras lead to a challenging motion transfer problem. In this paper, we propose an algorithm using time series analysis for motion transfer among multiple cameras. Specifically, we first identify seasonality in the motion data, and then build an additive time series model to extract patterns that could be transferred across different cameras. Our approach has a complete and clear mathematical formulation, and the algorithm is also efficient and interpretable. Through the experiment on real-world data, we demonstrate the effectiveness of our method. Furthermore, our motion transfer algorithm could combine with and facilitate downstream tasks, e.g., enhancing pose estimation on low-resolution (LR) videos with inherent patterns extracted from HR ones.Comment: 10 pages, 9 figure